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Abstract 

The performance of maximum-likelihood (ML) decoded binary linear block codes over the AWGN channel 
is addressed via the tangential-sphere bound (TSB) and two of its recent improved versions. The paper is 
focused on the derivation of the error exponents of these bounds. Although it was exemplified that some 
recent improvements of the TSB tighten this bound for finite-length codes, it is demonstrated in this paper 

■ that their error exponents coincide. For an arbitrary ensemble of binary linear block codes, the common 
\ value of these error exponents is explicitly expressed in terms of the asymptotic growth rate of the average 

■ distance spectrum. 

o 

^ ■ 1 Introduction 

O 



In recent years, much effort has been put into the derivation of tight performance bounds on 
the error probability of linear block codes under soft-decision maximum-likelihood (ML) decoding. 
During the last decade, this research work was stimulated by the introduction of various codes 
defined on graphs and iterative decoding algorithms, achieving reliable communication at rates 
close to capacity with feasible complexity. The remarkable performance of these codes at rates 
remarkably above the cut-off rate, makes the union bound useless for their performance evaluation. 
Hence, tighter performance bounds are required to gain some insight on the performance of these 
efficient codes. Improved upper and lower bounds on the error probability of linear codes under ML 
decoding are addressed in ^2] and references therein, and applied to various codes and ensembles. 

The tangential-sphere bound (TSB) [9] forms one of the tightest performance bounds for ML 
decoded linear block codes whose transmission takes place over the binary-input additive white 
Gaussian noise (BIAWGN) channel. The TSB was modified by Sason and Shamai jlOj for the 
analysis of the bit error probability of linear block codes, and was slightly refined by Zangl and 
Herzog ^H]- This bound only depends on the distance spectrum of the code (or the input-output 
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weight enumerating function (IOWEF) of the code for the bit-error analysis JO]), and hence, it can 
be applied to various codes or ensembles. The TSB falls within the class of upper bounds whose 
derivation relies on the basic inequality 

Pr(word error | Co) < Pr(word error, y E 1Z | Co) + Pr(y ^ 1Z \ Cq) (1) 

where Co is the transmitted codeword, y denotes the received vector at the output of the channel, 
and 1Z designates an arbitrary geometrical region which can be interpreted as a subset of the 
observation space. The basic idea of this bounding technique is to reduce the number of overlaps 
between the decision regions associated with the pairwise error probabilities used for the calculation 
of union bounds. This is done by separately bounding the error events for which the noise resides in 
a region TZ. The TSB, for example, uses a circular hyper-cone as the region 1Z. Other upper bounds 
from this family are addressed in ^[ Sections 3 and 4], ^H] and references therein. In ^5], Yousefi 
and Khandani prove that among all the volumes 1Z which possess some symmetry properties, the 
circular hyper-cone yields the tightest bound. This finding demonstrates the optimality of the 
TSB among a family of bounds associated with geometrical regions which possess some symmetry 
properties, and which are obtained by applying the union bound on the first term in the RHS of 
(|T]). In JHj) Yousefi and Khandani suggest to use the Hunter bound [H] (an upper bound which 
belongs to the family of second-order Bonf err oni- type inequalities) instead of the union bound. This 
modification should result in a tighter upper bound, and they refer to the resulting upper bound 
as the added hyper plane (AHP) bound. Yousefi and Mehrabian also apply the Hunter bound, 
but implement it in a quite different way in order to obtain an improved tangential-sphere bound 
(ITSB) which solely depends on the distance spectrum of the code. The tightness of the ITSB and 
the AHP bound is exemplified in |16l IT7] for some short linear block codes, where these bounds 
slightly outperform the TSB at the low SNR range. 

An issue which is not addressed analytically in [16II1T] is whether the new upper bounds (namely, 
the AHP and the ITSB) provide an improved lower bound on the error exponent as compared to 
the error exponent of the TSB. In this paper, we address this question, and prove that the error 
exponents of these improved tangential-sphere bounds coincide with the error exponent of the TSB. 
We note however that the TSB fails to reproduce the random coding error exponent, especially for 
high-rate linear block codes 

This paper is organized as follows: The TSB ([0], PHI), the AHP bound 16^ and the ITSB [Tlj 
are presented as a preliminary material in Section |^1 In Section |3J we derive the error exponents 
of the ITSB and the AHP, respectively and state our main result. We conclude our discussion in 
Section |IJ An Appendix provides supplementary details related to the proof of our main result. 

2 Preliminaries 

We introduce in this section some preliminary material which serves as a preparatory step towards 
the presentation of the material in the following section. We also present notation from which 
is useful for our analysis. The reader is referred to |12| I18j which introduce material covered in 
this section. However, in the following presentation, we consider boundary effects which were not 
taken into account in the original derivation of the two improved versions of the TSB in |16j-|18j). 
Though these boundary effects do not have any implication in the asymptotic case where we let 
the block length tend to infinity, they are addressed in this section for finite block lengths. 
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2.1 Assumption 



Throughout this paper, we assume a binary-input additive white Gaussian noise (AWGN) channel 
with double-sided power spectral density of The modulation of the transmitted signals is 
antipodal, and the modulated signals are coherently detected and ML decoded (with soft decision). 



2.2 Tangential-Sphere Bound 

The TSB forms an upper bound on the decoding error probability of ML decoding of linear block 
code whose transmission takes place over a binary-input AWGN channel [HI El- Consider an (n, k) 
linear block code C of rate R = ^ bits per channel use. Let us designate the codewords of C 
by {cj}, where i = 0, 1, . . . , 2 k — 1. Assume a BPSK modulation and let Sj G {+y/~E^, —^fE~ s } n 
designate the corresponding equi-energy, modulated vectors, where E s designates the transmitted 
symbol energy. The transmitted vectors {sj} are obtained from the codewords {cj} by applying the 
mapping Sj = (2cj — 1)\/T^, so their energy is nE s . Since the channel is memoryless, the received 
vector y = 7/2, • • • , y n ), given that Sj is transmitted, can be expressed as 

Vj = S i,j + Z j ' i = 1 > 2 , ( 2 ) 

where 81 is the j th component of the transmitted vector Sj, and z = (z\, Z2, ■ ■ ■ , z n ) designates an 
re-dimensional Gaussian noise vector which corresponds to re orthogonal projections of the AWGN. 
Since z is a Gaussian vector and all its components are un-cor related, then the n components of z 
are i.i.d., and each component has a zero mean and variance a 2 = 

Let E be the event of deciding erroneously (under ML decoding) on a codeword other than the 
transmitted codeword. The TSB is based on the central inequality 

Pr(£|c ) < Pr(£, y G ft|c ) + Pr(y $ ft|c ) (3) 

where 1Z is an n-dimensional circular cone with a half angle 9 and a radius r, whose vertex is 
located at the origin and whose main axis passes through the origin and the point corresponding 
to the transmitted vector (see Fig^). The optimization is carried over r (r and 9 are related as 
shown in Fig. Let us designate this circular cone by C n (9). Since we deal with linear codes, 
the conditional error probability under ML decoding does not depend on the transmitted codeword 
of the code C, so without any loss of generality, one can assume that the all-zero codeword, so, is 
transmitted. Let z\ be the radial component of the noise vector z (see Fig. ^) so the other n — 1 
components of z are orthogonal to the radial component z\. From Fig. ^ we obtain that 



r = \J nE s tan 9 

r Zl = ( \fnE s - z\ J tan ( 



VnE~ s - z\ 5 k 



(3 k ( Zl ) = ( VnK ~ zi ) tanC = ^^=^± (4) 

'nE R - % 
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The random variable Y = Y17=2 z \ ^ s X 2 distributed with n — 1 degrees of freedom, so its pdf is 



given by 



n-3 V 

JY{y) = —^1 , y>0 (5 

2— a n - x T (Sfl) 
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Figure 1: The geometric interpretation of the TSB. 



where U designates the unit step function, and the function T is the complete Gamma function 

/•oo 

T(x) = / f^dt, Real(x) > 0. (6) 
Jo 

Conditioned on the value of the radial component of the noise, z±, let E(z±) designate the 
decoding error event. The conditional error probability satisfies the inequality 

Pr(£7(zi) | z±) < Pr (E(zi), y € C n {9) \ Zl ) + Pr (y i C n {9) \ Zl ) (7) 

The conditional error event E{z\) can be expressed as a union of pairwise error events, so 



Pr(£7(zi),y G C n {6) \ Zl ) = Pr ^ [J E ^( Zl ), y € C„(0) | ^ , M = 2 k 



(8) 



where Eo-+i(z\) designates the event of error had the only codewords been Co and Cj, given the 
value z\ of the radial component noise in Fig. and M = 2 k denotes the number of codewords of 
the code C. We note that for BPSK modulation, the Euclidean distance between the two signals s% 
and so is directly linked to the Hamming weight of the codeword Cj. Let the Hamming weight of 
Cj be h, then the Euclidean distance between so and Sj is equal to 6^ = 2^fhE s . Let {A^} be the 
distance spectrum of the linear code C, and let Eh(z x ) be the event of deciding under ML decoding 
in favor of other codeword Cj whose Hamming weight is h, given the value of z x . By applying the 
union bound on the RHS of (JHJ), we get 

n 

Pr(%),yG C n {9) | z x ) < ^2A h Px(E h {zi),y G C n {9) \ z x ). (9) 

h=X 

Combining (J7J) and ® gives 

Pr [E{zi) \z x ) < { A h Pr (E h (z x ),y G C n (9) \z 1 )} + Pr (y $ C n (6) \ z x ) . (10) 
h 

The second term in the RHS of (jl()j) is evaluated from Q 

Pr(y i C n (9) | z x ) = Vx[Y>r 2 zl \ z x ) 



fv{y)dy 

,2 

„ n— 2 V _ 

*> y— e -^U(y) 

\ 2V CT -ir(^ 



dy. (11) 



7 (a,3;) = -i-/ Zr B - 1 e~*£ft, a > 0, x > (12) 



This integral can be expressed in terms of the incomplete Gamma function 

1 

r(a; jo 
and it is transformed to 

Pr(y C B (0) | z x ) = 1 - 7 ^) . (13) 

Let ^2 designate the tangential component of the noise vector z, which is on the plane that contains 
the signals So, s, and the origin of the space, and orthogonal to z x (see Fig. Referring to the 
first term in the RHS of (|10|) , it follows from the geometry in Fig. ^ that if z x < y/nE s then 

Pr(E h (z x ),y G C n {6) \ z x ) = Pv(E h (z x ),Y < r\ \ z x ) 

= Pr (p h (z x ) <z 2 <r Zl ,Y< r\ \ z x ) . (14) 

Let V = X^r=3 z h t nen V = Y — z\. If z x < \fnE~ s , then we obtain the equality 

PT(E h (z x ),y G C n (0) | z x ) = Pr {(3 h (z x ) <z 2 <r Zl ,V< r 2 zi - z\ \ z x ) . (15) 
The random variable V is x 2 distributed with n — 2 degrees of freedom, so its pdf is 

7i-4 y 

fv{v) = -^=g — — , v > (16) 

2 — a n-2 F n^2) 
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and since the random variables V and Z 2 are statistically independent, then if z\ < \/nE B 



,2 

i2_ ,jz ,2 



PT(E h {z 1 ),yeC n (e)\z 1 )= 1 / • f v (v)dvdz 2 . (17) 

Jf3 h (z-i_) v lira Jo 

In order to obtain an upper bound on the decoding error probability, Pr(E'), one should apply the 
statistical expectation operator on the RHS of (|1U[) w.r.t. the radial noise component z\. Referring 
to the upper half azimuthal cone depicted in Fig. Q which corresponds to the case where the radial 
noise component satisfies the condition z\ < \/nE s , the inequality (3h{ z \) < r z 1 holds for the values 
of h for which ^ < ah where 



W'-Jr (18) 



On the other hand, if z\ > \/nE s , the range of integration for the component noise z 2 is /3h(zi) < 
z 2 < — r zi which is satisfied for all values of h (since for z\ > \/nE s , we get from (jlj) that r Zl < 
and f3h(z\) < 0, so the inequality Ph{zi) < — r Zl holds in this case for all values of h). Since 
Z\ ~ N(0,a 2 ) where a 2 = 4^, then the probability that the Gaussian random variable Z\ exceeds 
^/nE s is equal to 

' \fnK\ / /2nRE h 



Q 



<J J \\ N J 

This results in the following upper bound on the decoding error probability under ML decoding 



2 2 z 2 



V27R7 I f- 1 L J/3 h ( 2l ) V2vrcr 
v /i:^f<a,,. 




+l-7(^,&)k + Qfx/^)- (19) 



The upper bound (j!9j) is valid for all positive values of r. Hence, in order to achieve the tightest 
upper bound of the form ()19j) one should set to zero the partial derivative of the RHS of (|19|) w.r.t. 
r zi . After straightforward algebra the following optimization equation for the optimal value of r is 
obtained [SJ: 

,ra— 3 



sin 



<« h " (20) 



cos 



2a h 



where ah is given in (|18|) . A proof for the existence and uniqueness of a solution r to the optimization 
equation (|20|) was provided in |11| Appendix B], together with an efficient algorithm to solve this 
equation numerically. In order to derive an upper bound on the bit error probability, let A Wj h 
designate the corresponding coefficient in the IOWEF which is the number of codewords which are 
encoded by information bits whose number of ones is equal to w (where < w < nR) and whose 
Hamming weights (after encoding) are equal to h, and define 

nR 

4>-ES)^' h = 0,...,n. (21) 
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In Appendix C], Sason and Shamai derive an upper bound on the bit error probability by 
replacing the distance spectrum {Ah} in (jT?3|) and (f277|) with the sequence {-A^}, and show some 
properties on the resulting bound on the bit error probability. 



2.3 Improved Tangential-Sphere Bound 

In ^7j, Yousefi and Mehrabian derive a new upper bound on the block error probability of binary 
linear block codes whose transmission takes place over a binary-input AWGN channel, and which 
are coherently detected and ML decoded. This upper bound, which is called improved tangential- 
sphere bound (ITSB) is based on inequality ©, where the region 1Z is the same as of the TSB (i.e., 
an ra-dimensional circular cone). To this end, the ITSB is obtained by applying a Bonf err oni- type 
inequality of the second order [3l|S] (instead of the union bound) to get an upper bound on the joint 
probability of decoding error and the event that the received vector falls within the corresponding 
conical region around the transmitted signal vector. 

The basic idea in relies on Hunter's bound which states that if {Ei}f£± designates a set of 
M events, and Ef designates the complementary event of Ei, then 



= Pr(£ x ) + Pr(E 2 n E{) + . . . + Pv(E M n Efc^ ...HE?) 

M 

^Pr^O+^Pr^nE?). (22) 



i=2 



where the indices i £ {1, 2, ... i — 1} are chosen arbitrarily for i S {2, . . . , M}. Clearly, the upper 
bound (|22|) is tighter than the union bound. The LHS of (|22jl is invariant to the ordering of the 
events (since it only depends on the union of these events), while the RHS of (|22|) depends on this or- 
dering. Hence, the tightest bound of the form (|22j) is obtained by choosing the optimal indices order- 
ing i € {1,2, ... , M} and i £ {1, 2, . . . ,i— 1}. Let us designate by n(l, 2, . . . , M) = {m, iT2, ■ ■ ■ , ttm} 
an arbitrary permutation among the Ml possible permutations of the set {1, 2, . . . , M} (i.e., a per- 
mutation of the indices of the events E\ to Em), and let A = (A2, A3, . . . Am) designate an arbitrary 
sequence of integers where Aj 6 {^1,^2, ■ ■ ■ TTj-i}- Then, the tightest form of of the bound in (|22j) 
is given by 

Prl I 1^1 ^min^Pr^J + VPr^n^jL (23) 



' M \ ( M \ 

\JeA <min Pr^J+^Pr^n^) 

vi=l / ' I i=2 ) 



Similar to the TSB, the derivation of the ITSB originates from the upper bound (JJJ) on the 
conditional decoding error probability, given the radial component (z\) of the noise vector (see 
Fig. P). In it is proposed to apply the upper bound ([23]) on the RHS of (jHJ which for an 
arbitrary permutation {tti,tt2, ■ ■ ■ , ^m} and a corresponding sequence of integers (A2, A3, . . . \m-i) 
as above, gives 



Pr ^ J E o^y e C n (0) I z^j < minjpr^o^.y 6 C n {6) \ Zl ) 

M-l } 

+ ^ Pr(i?o^„^ A! ,yG C n (9) \ z x ) (24) 



i=2 
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where i?o— »j designates the pairwise error event where the decoder decides on codeword Cj rather 
than the transmitted codeword Co- As indicated in |15| 117] . the optimization problem of (|24j) 
is prohibitively complex. In order to simplify it, Yousefi and Mehrabian suggest to choose tti = 
\ = ^min for all £ = 2, . . . , M — 1, where i m \ n designates the index of a codeword which is closest 
(in terms of Euclidian distance) to the transmitted signal vector so- Since the code is linear and 
the channel is memoryless and symmetric, one can assume without any loss of generality that 
the all-zero codeword is transmitted. Moreover, since we deal with antipodal modulation, then 
^H(cj min ) = dmin where d m \ n is the minimum distance of the code. Hence, by this specific choice 
of 7Ti and A (which in general loosen the tightness of the bound in H24j) ). the ordering of the 
indices {tt2, ■ ■ ■ ,ttm} is irrelevant, and one can omit the optimization over II and A. The above 
simplification results in the following inequality: 

Pr(E\ Zl ) <Pr(^w,Y € C n {9) | z x ) 

M-l 

+ Pr^^^jG C n (6) | z x ) +Pr(y $ C n (9) | z x ) . (25) 

i=2 

Based on Fig. ^ the first and the third terms in the RHS of (|25|) can be evaluated in similarity 
with the TSB, and we get 

Pr(Eo^ mb ,yG C„(0) | zi) = Pr(/3 min (zi) < z 2 < r zi , V < r\ - z\\ z x ) (26) 

Pr(y i C n {9) | zO = 1 - 7 ^— , (27) 

where 

/3 min (^i) = (V^Es ~ zA J dm f , (28) 
V / V n - d min 

Z2 is the tangential component of the noise vector z, which is on the plane that contains the signals 

so, Sj min and the origin (see Fig. P), and the other parameters are introduced in 

For expressing the probabilities of the form Pr(Eo^i, . ,y G C n {9) \ z\) encountered in 

the RHS of ()25|) . we use the three-dimensional geometry in Fig. 121(a). The BPSK modulated 

signals so , and Sj are all on the surface of a hyper-sphere centered at the origin and with radius 

^JnE s . The planes Pi and P2 are constructed by the points (o, so,Sj) and (o, so,Sj), respectively. 

In the derivation of the ITSB, Yousefi and Mehrabian choose sj to correspond to codeword Cj with 

Hamming weight d m { n . Let z' 3 be the noise component which is orthogonal to z\ and which lies on 

the plane P2 (see Figl^a). Based on the geometry in Fig. |2J-a (the probability of the event Eq^- 

is the probability that y falls in the dashed area) we obtain the following equality if z\ < \/nE s : 

= Pr (ft(zi) < z 2 < r Zl , -r Zl < z' 3 < /? min (zi), Y < r\ \ z x ) . (29) 

Furthermore, from the geometry in Fig. |2j-b, it follows that 

Z3 = Z3 sin 4> + Z2 cos (p. (30) 

where Z3 is the noise component which is orthogonal to both z\ and Z2, and which resides in the 
three-dimensional space that contains the signal vectors sq, Sj, Sj min and the origin. Plugging (|30j) 
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into the condition — r Zx < z' 3 < P m m(zi) in (|29j) yields the condition — r Zl < z% < min{l(zi, Z2),r Zl } 
where 

U \ Pmin(zi) - pz 2 . , 

l(zi,z 2 ) = r — (31) 

V 1 - P 

n 

and p = cose/) is the correlation coefficient between planes Pi and Pi. Let W = ^^zf, then if 

8=4 



Z\ < y/nE s 

Pr(^,^o%w,yec^)Ui) 

= Pr (/3i(zi) < z 2 < r w , -r Zl < z 3 < min{l(z 1 , z 2 ),r Zl }, W <r\- z\- z\ \ z\) . (32) 

The random variable W is Chi-squared distributed with n — 3 degrees of freedom, so its pdf is given 
by 

n — 5 _ VJ 

Since the probabilities of the form Pr(_Eo->i, . ,y G C n (9) \ z\) depend on the correlation 

coefficients between the planes (o, so,Sj min ) and (o,So,Sj), the overall upper bound requires the 
characterization of the global geometrical properties of the code and not only the distance spectrum. 
To circumvent this problem and obtain an upper bound which is solely depends on the distance 
spectrum of the code, it is suggested in JJj to loosen the bound as follows. It is shown [TTfl 
Appendix B] that the correlation coefficient p, corresponding to codewords with Hamming weights 
di and dj satisfies 



. . , djdj (n- dj){n - dj) \ min(dj, dj)[n - max(^, dj)] 

' U1U ^ l/ {n-di){n-djy\j ddj }~ P ~ v / d i d j (n-d i )(n-d j ) ' [ ' 

Moreover, the RHS of 1)32(1 is shown to be a monotonic decreasing function of p (see |171 Ap- 
pendix 1]). Hence, one can omit the dependency in the geometry of the code (and loosen the upper 
bound) by replacing the correlation coefficients in (|32|) with their lower bounds which solely depend 
on the weights of the codewords. In the derivation of the ITSB, we consider the correlation coef- 
ficients between two planes which correspond to codewords with Hamming weights di = h, h> n 
and dj = d m \ a . Let 



Ph = ~ mm 



hd min (n-h)(n- d min ) 



(n — h) (n — dmin) V ^d 



mm 



hd m \n /or\ 

(35) 



(n -h)[n- dmin)' 



where the last equality follows directly from the basic property of d m { n as the minimum distance 
of the code. From (|25) ) -(|2fij ) and averaging w.r.t. Z±, one gets the following upper bound on the 
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decoding error probability: 



Pt{E) < Pr U < y/nE a , /3 min (zi) <z 2 <r Zl ,V< r\ - z\ 



+ A h?r[zi < \A^> <z 2 < r Zl , 



r Zl < z 3 < mm{l h (z 1 ,z 2 ),r Zl },W < r\ - z\ - zfj 



+ Pr (zx < Vn£^, Y > r 2 Zi j + Pr(zi > y/riE B ) 
where the parameter lf l (zi,z 2 ) is simply l(zi,z 2 ) in (|3*Tj) with p replaced by ph, i.e., 

Anin(^l) - PhZ2 



k(zi,z 2 ) 



(36) 



(37) 



Using the probability density functions of the random variables in the RHS of ()36|) , and since the 
random variables Z%, Z 2 , Z 3 and W are statistically independent, the final form of the ITSB is given 
by 



Po < 



fz 2 (z 2 ) 



zi ^2 



+ £ 

h:/3 h (z 1 )<r zi 



fv(v)dv ■ dz 2 

m'm{l h (z 1 ,Z2),r zl } 



Ph{zi) J-r n 



+ 1-7 



n — 1 r; 



2 '2a 2 



f Zl {zi)dzi + Q 



fz 2 ,z 3 (z 2 ,z 3 ) 



2nRE h \ 
No J ' 



fw(w)dw ■ dz 2 ■ dz 3 J 
(38) 



Note that V = J27=3 z f an< ^ W = Ya=4 z i are Chi-squared distributed with (n — 2) and (n — 3) 
degrees of freedom, respectively. 



2.4 Added-Hyper-Plane (AHP) Bound 

In [121, Yousefi and Khandani introduce a new upper bound on the ML decoding block error 
probability, called the added hyper plane (AHP) bound. In similarity with the ITSB, the AHP 
bound is based on using the Hunter bound (|22|) as an upper bound on the LHS of Q, which results 
in the inequality (|24|) . The complex optimization problem in (|24j) . however, is treated differently. 
Let us denote by I w the set of the indices of the codewords of C with Hamming weight w. For 
i £ {1,2, ... , M} \ I W , let {ji} be a sequence of integers chosen from the set X w . Then the following 
upper bound holds 

Pr(£(*i),y€C n (0)|*i) 

< min | Pr J {E ^}, y 6 C n {9) \ zA + £ Pr {E^E^y G C n (0) | *l) I . 
W ' w [ \jeiu, J ie{i,...,M-i}\i w J 

(39) 
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The probabilities inside the summation in the RHS of (|39|) are evaluated in a similar manner to the 
probabilities in the LHS of (j29|h From the analysis in Section \'2.'A\ and the geometry in Fig. [21- (b) , 
it is clear that the aforementioned probabilities depend on the correlation coefficients between the 
planes (o, so,Sj) and (o, so,SjJ. Hence, in order to compute the upper bound (|39|) . one has to 
know the geometrical characterization of the Voronoi regions of the codewords. To obtain an upper 
bound requiring only the distance spectrum of the code, Yousefi and Khandani suggest to extend 
the codebook by adding all the (™) — A w n-tuples with Hamming weight w (i.e., the extended code 
contains all the binary vectors of length n and Hamming weight w). Let us designate the new code 
by C w and denote its codewords by 

cf, ieU,l,...,M+r)-A w -l 

The new codebook is not necessarily linear, and all possible correlation coefficients between two 
codewords with Hamming weight i, where i E {cW n , . . . d max }, and w are available. Thus, for 
each layer of the codebook, one can choose the largest available correlation 1 p with respect to any 
possible n-tuple binary vector of Hamming weight w. Now one may find the optimum layer at 
which the codebook extension is done, i.e., finding the optimum w G {1, 2, ... n} which yields the 
tightest upper bound within this form. We note that the resulting upper bound is not proved to 
be uniformly tighter than the TSB, due to the extension of the code. The maximum correlation 
coefficient between two codewords of Hamming weight di and dj is introduced in the RHS of (|34[) 
(see jlEl)- Let us designate the maximal possible correlation coefficient between two n-tuples with 
Hamming weights w and h by p w h , i.e., 

mm(h,w)[n - max(h,w)] 

Pw,h = , = , w^h. (40) 

sj aw{n — n)[n — w) 

By using the same bounding technique of the ITSB, and replacing the correlation coefficients with 
their respective upper bounds, p W) h, P^|> gets the form 

Pr (E( Zl ),y G C n {8) \ Zl ) < mm J Pr \J {E ^},y G C n {9) \ z x J 

+ A h Pi(Y < r 2 zv [5 h { Zl ) < z 2 , z 3 < l W)h (z u z 2 ) \ zi) \ 

(41) 

where 

, / \ Pw(zi) ~ Pw,hZ2 /. v 
L,h{Zl,Z 2 ) = ; — • (42) 

1 - pLh 



1 The RHS of l)39|l is a monotonically decreasing function of p, as noted in |17j . 
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Now, applying Hunter bound on the first term in the RHS of (|41|) yields 

pr I U y G c «w 1 21 

\j:w n (cj)=w 

CM 

< Pr(E _/ ,y G C„(0) | zi) + Pr(E ^, G C n {6) \ z x ) (43) 

i=l 

where {k}, i G {0, 1, . . . , (™) — l} is a sequence which designates the indices of the codewords of 
C w with Hamming weight w with an arbitrary order, and l{ G (Zo, ii, • • • , Zj-i)- In order to obtain 
the tightest bound on the LHS of (|4$|) in this approach, one has to order the error events such that 
the correlation coefficients which correspond to codewords c^. and c? get their maximum available 
value, which is 1 — w (™- w ) P^l Appendix D]. Let us designate this value by p WjW i.e., 

TL 

Pw,w = 1 7 r , w (£ {0, n}. 

win — w) 



Hence, based on the geometry in Fig. |2J if Z\ < \/nE s , we can rewrite (|43f) as 
Prl |J Eo^y E C n (9) \ Zl 

\j:w s (cf)=w , 

< Pr (p w ( Zl ) <Z2<r Zl ,V< r\ - z\ \ z x ) 

+ y^j - 1 Pr (P w (zi) < z 2 < r Zl ,-r Zl < z 3 < min{l W)W (z 1 , z 2 ),r zl },W < r\ - z\- z\\ Z\) 

(44) 

where 

L, w {zi,z 2 ) = . — . (45) 

y I ~~ Pw,w 

By replacing the first term in the RHS of (|41|) with the RHS of (|44j) . plugging the result in ((7J) and 
averaging w.r.t. Z\ finally gives the following upper bound on the block error probability: 

Pr(E) < nun j Pr < y^k, Pw(zi) < z 2 < r Zl , V < r\ - z^ 

+ (™\ Pr (z\ < y / nE~ s , (3 w (zi) < z 2 < r Zl ,-r zl < z 3 < mm{l w , w (zi, z 2 ),r Zl }, W < r Zl - z 2 - z 

+ ^ AhPr ( Zl - V nE s, Ph(zi) < z 2 < r zl ,-r Zl < z 3 < min{l W)h ( Zl ,z 2 ),r zl },W < r\ - z\ - 

+ Px(z 1 <y/^ B ,Y>f^j+Pi(zi>y/nE^). (46) 
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Rewriting the RHS of 1)461) in terms of probability density functions, the AHP bound gets the form 



P e < min 



r \~ z l 



fz^W) \ fv{v)dv ■ dz 2 

PwM Jo 



+ 



■»' 



r Zl rmin{l W!W (zi,Z2),r zl } 



fz 2 ,Z 3 (z2,Z3) 



r z 1 z 2 z 3 



fw(w)dw • dz2 ■ dz% 







+ £ 



h : P h (zi) < r zi 

h W 



A h I / fz 2 ,z 3 (z2,Z3) f w (w)dw ■ dz 2 ■ dz 3 



,'n—l rj^ 



fz 1 (zi)dz 1 > + Q 



2nRE h \ 



No ) 



(47) 



where V and W are introduced at the end of Section \'2.'A\ (after Eq. (jSHJ)), and the last term in ()47j) 
follows from (|13|) . 



3 The Error Exponents of the ITSB and AHP Bounds 

The ITSB and the AHP bound were originally derived in |16L I17j as upper bounds on the ML 
decoding error probability of specific binary linear block codes. In the following, we discuss the 
tightness of the new upper bounds for ensemble of codes, as compared to the TSB. The following 
lemma is also noted in |17j . 

Lemma 1. Let C be a binary linear block code, and let us denote by ITSB(C) and TSB(C) the 
ITSB and TSB, respectively, on the decoding error probability of C. Then 

ITSB(C) < TSB(C). 

Proof. Since Pr(A, B) < Pr(^4) for arbitrary events A and B, the lemma follows immediately by 
comparing the bounds in the RHS of ()1U|) and 1)25)1 . reffering to the TSB and the ITSB, respectively. 

□ 

Corollary 1. The ITSB can not exceed the value of the TSB referring to the average error prob- 
ability of an arbitrary ensemble of binary linear block codes. 

Lemma 2. The AHP bound is asymptotically (as we let the block length tend to infinity) at least 
as tight as the TSB. 

Proof. To show this, we refer to (|46[). where we choose the layer w at which the extension of the 
code is done to be n. Hence, the extended code contains at most one codeword with Hamming 
weight n more than the original code, which has no impact on the error probability for infinitely 
long codes. The resulting upper bound is evidently not tighter than the AHP (which carries an 
optimization over w), and it is at least as tight as the TSB (since the joint probability of two events 
cannot exceed the probabilities of these individual events). □ 
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The extension of Lemma |2] to ensembles of codes is straightforward (by taking the expectation 
over the codes in an ensemble, the same conclusion in Lemma 01 holds also for ensembles). From 
the above, it is evident that the error exponents of both the AHP bound and the ITSB cannot be 
below the error exponent of the TSB. In the following, we introduce a lower bound on both the 
ITSB and the AHP bound. It serves as an intermediate stage to get our main result. 

Lemma 3. Let C designate an ensemble of linear codes of length n, whose transmission takes place 
over an AWGN channel. Let Ah be the number of codewords of Hamming weight h, and let Ec 
designate the statistical expectation over the codebooks of an ensemble C. Then both the ITSB 
and AHP upper bounds on the average ML decoding error probability of C are lower bounded by 
the function t/j(C) where 

^(C) = mm |ec [Pr [zx < s/^E a , f3 w { Zl ) <z 2 <r Zl ,V< r 2 Zi - zf) 

+ J2<A h Pi(zi < ^E B , p h {z x ) < z 2 <r Zl , 
h ^ 

- r Zl < z 3 < mm{l wA (z 1 ,z 2 ),r Zl },W < r\ - z\ - zf\ 



+ PT{z 1 <^E s ,Y>r 2 zi ) } (48) 



and l w ,h( z i,z 2 ) is defined in (|42|). 

Proof. By comparing 1)46(1 with (|48|). it is easily verified that the RHS of (|48jl is not larger than 
the RHS of 1)46(1 (actually, the RHS of (|48|l is just the AHP without any extension of the code). 
Referring to the ITSB, we get 

ITSB(C) = E c [Pr (z x < V^E S , /W*i) <z 2 <r Zl ,V< r\ - z^) 

+ Yj\ A h**{*\ ^ V™E~s, 0h(zi) <Z2 <r Zl , 
h 

- r zi < z 3 < mm{l h (zi,z 2 ),r zi }, W <r\- z\- z^\ 
+ Pr (z x < v^F s , Y > r^)] + Pr (zx > y^F s 



|e c [l 



> min Ec Pr U< y/^E B , f3 w (zx) < z 2 < r zi , V < r\ 



z 2 



+ A fc Pr(a;i < ^nE s , p h {zi) <z 2 < r zi , 
h 

- r zi < z 3 < mm{l Wjh (zx, z 2 ),r Zl },W < r\ - z\ - z|) j 

+ Pr [zx < y/^E s , Y > r^)] | + Pr (z x > y/^E^ 
> V(C). (49) 
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The first inequality holds since the ITSB is a monotonically decreasing function w.r.t. the corre- 
lation coefficients (see Appendix IA.3|) , The equality in (|49|) is due to the linearity of the function 
in (|49[) w.r.t. the distance spectrum, on which the expectation operator is applied, and the last 
transition follows directly from (|48jl. □ 

In ^2] and [E], the RHS of (|46j) and (|36|) . respectively, were evaluated by integrals, which results 
in the upper bounds ijlTI) and (J3EJ)- In ^ Section D], Divsalar introduced an alternative way to 
obtain a simple, yet asymptotically identical, version of the TSB by using the Chernoff bounding 
technique. Using this technique we obtain the exponential version of ip(C). In the following, We 
use the following notation pP: 



a E s A h A / 6 A \n(A h ) 

C =TF' S =Z> A =\/i I> 



No n \ 1 — 5 n 

where for the sake of clear writing we denote the average spectrum of the ensemble by A^. We now 
state the main result of this paper. 

Theorem 1. (The error exponent of the AHP and the ITSB bounds coincide with the 
error exponent of the TSB) The upper bounds ITSB, AHP and the TSB have the same error 
exponent, which is 

E{c) = min ( I In f 1 - 7 + je^A + 1 (50) 



(51) 



0«Kl\2 \ ' ' J I + 7A 2 

where 



7 = l{S) 

and 



+ (1 + C) 2 -1-(1 + C) 



co (£) 

co(^(l-e- 2 ^))^. (52) 



Proof. The exponential version of ip(C) in (|48|) is identical to the exponential version of the TSB 
(see Appendices IA.1I and IA.2|) . Since tp(C) does not exceed the AHP and the ITSB, this implies 
that the error exponents of the AHP and the ITSB are not larger than the error exponent of the 
TSB. On the other hand, from Lemmas ^ and |^1 it follows that asymptotically, both the AHP and 
the ITSB are at least as tight as the TSB, so their error exponents are at least as large as the 
error exponent of the TSB. Combining these results we obtain that the error exponent of the ITSB, 
AHP and the TSB are all identical. In pQ, Divsalar shows that the error exponent of the TSB is 
determined by (|50j) - (|52|) . which concludes the proof of the theorem. □ 

Remark 1. The bound on the bit error probability in 11 is exactly the same as the TSB on 
the block error probability by Poltyrev [S], except that the average distance spectrum {^4^} of the 
ensemble is now replaced by the sequence {A' h } where 

nR 

A! h = Y^{^) Aw >h, he{0,...,n} 

w=0 
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and A w h denotes the average number of codewords encoded by information bits of Hamming weight 
w and having a Hamming weight (after encoding) which is equal to h. Since = ^™=o Au,/i> 
then 

-^<A' h <A h , he{0,...,n}. 
nix 

The last inequality therefore implies that the replacement of the distance spectrum {Ah} by {A' h } 
(for the analysis of the bit error probability) does not affect the asymptotic growth rate of r(5) 
where 5 = ^, and hence, the error exponents of the TSB on the block and bit error probabilities 
coincide. 

Remark 2. In ^H], Zangl and Herzog suggest a modification of the TSB on the bit error probability. 
Their basic idea is tightening the bound on the bit error probability when the received vector y 
falls outside the cone 1Z in the RHS of © (see Fig. [Q). In the derivation of the version of the 
TSB on the bit error probability, as suggested by Sason and Shamai the conditional bit error 
probability in this case was upper bounded by 1, where Zangl and Herzog 19. refine the bound and 
provide a tighter bound on the conditional bit error probability when the vector y falls in the bad 
region (i.e., when it is outside the cone in Fig.^). Though this modification tightens the bound on 
the bit error probability at low SNR (as exemplified in f° r some short linear block codes), it has 
no effect on the error exponent. The reason is simply because the conditional bit error probability 
in this case cannot be below (i.e., one over the dimension of the code), so the bound should still 
possess the same error exponent. This shows that the error exponent of the TSB versions on the 
bit error probability, as suggested in and JH]> coincide. 

Corollary 2. The error exponents of the TSB on the bit error probability coincides with the error 
exponent of the TSB on the block error probability. Moreover, the error exponents of the TSB on 
the bit error probability, as suggested by Sason and Shamai and refined by Zangl and Herzog 
|19j . coincide. The common value of these error exponents is explicitly given in Theorem ^ 

4 Summary and Conclusions 

The tangential-sphere bound (TSB) of Poltyrev [2] often happens to be the tightest upper bound on 
the ML decoding error probability of block codes whose transmission takes place over a binary-input 
AWGN channel. However, in the random coding setting, it fails to reproduce the random coding 
error exponent jH] while the second version of the Duman and Salehi (DS2) bound does pi I12j. 
The larger the code rate is, the more significant becomes the gap between the error exponent of 
the TSB and the random coding error exponent of Gallager [S] (see Fig. |3 and the plots in 
Figs. 2-4]). In this respect, we note that the expression for the error exponent of the TSB, as 
derived by Divsalar , is significantly easier for numerical calculations than the original expression 
of this error exponent which was provided by Poltyrev 0J Theorem 2]. Moreover, the analysis made 
by Divsalar is more general in the sense that it applies to an arbitrary ensemble, and not only to 
the ensemble of fully random block codes. 

In this paper, we consider some recently introduced performance bounds which suggest an 
improvement over the TSB. These bounds rely solely on the distance spectrum of the code (or their 
input-output weight enumerators for the analysis of the bit error probability). We study the error 
exponents of these recently introduced bounding techniques. This work forms a direct continuation 
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to the derivation of these bounds by Yousefi et al. [1611171 ITH] who also exemplified their superiority 
over the TSB for short binary linear block codes. 

Putting the results reported by Divsalar with the main result in this paper (see Theorem 
we conclude that the error exponents of the simple bound of Divsalar pQ, the first version of Duman 
and Salehi bounds 2 , the TSB 9 and its improved versions by Yousefi et al. ^3EEE] all coincide. 
This conclusion holds for any ensemble of binary linear block codes (e.g., turbo codes, LDPC codes 
etc.) where we let the block lengths tend to infinity, so it does not only hold for the ensemble of 
fully random block codes (whose distance spectrum is binomially distributed). Moreover, the error 
exponents of the TSB versions for the bit error probability, as provided in [111 I19j . coincide and 
are equal to the error exponent of the TSB for the block error probability. The explicit expression 
of this error exponent is given in Theorem Q and is identical to the expression derived by Divsalar 

for his simple bound. Based on Theorem ^ it follows that for any value of SNR, the same 
value of the normalized Hamming weight dominates the exponential behavior of the TSB and its 
two improved versions. In the asymptotic case where we let the block length tend to infinity, the 
dominating normalized Hamming weight can be explicitly calculated in terms of the SNR; this 
calculation is based on finding the value of the normalized Hamming weight 5 which achieves the 
minimum in the RHS of (|50j) . where this value clearly depends on the asymptotic growth rate of 
the distance spectrum of the ensemble under consideration. A similar calculation of this critical 
weight as a function of the SNR was done in [3], referring to the ensemble of fully random block 
codes and the simple union bound. 

In a companion paper Jl], new upper bounds on the block and bit error probabilities of linear 
block codes are derived. These bounds improve the tightness of the Shulman and Feder bound |I2f 
and therefore also reproduce the random coding error exponent. 
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Appendix 



A.l The exponent of ip(C) 

In the following, the exponential behavior of the RHS of l)48j) is obtained by using the Chernoff 
bounding technique for tp(C). 

Note that the geometrical region of the TSB corresponds to a double sided circular cone. For 
the derivation of the bound for the single cone, we have put the further restriction z\ < \/nE s , but 
since z\ ~ iV(0, then this boundary effect does not have any implication on the exponential 
behavior of the function ip(C) for large values of n (as also noted in [l] p. 23]). To simplify the 
analysis, we therefore do not take into consideration of this boundary effect for large values of n. 
Let vp(C) designate the function which is obtained by removing the event Z\ < \JnE s from the 
expression for ip(C) (see the RHS of (J1%|0 . 

Let us designate the normalized Gaussian noise vector by v, i.e., (y\, . . . , v n ) = \[-^{zi, ■ ■ ■ , z n ), 

and define rj = tan 2 6. The Gaussian random vector has n orthogonal components which are 
therefore statistically independent. From (J1J and ()42j) . the following equalities hold for BPSK 
modulated signals: 



r = \j2ncr) 
r »i = VV (V2nc - v\ 



h 



n — h 

lw,h(Vl,V2) = , — • (A.l) 

Hence, we obtain from (|48|) and the above discussion 

^(C) = min | Pr ^ v} < r 2 vi , v 2 > ^{v^ 

n / n \ 

+ ^2 Ah Pr I ^2 v\ < r 2 ui ,u 2 > 0h(^l),^3 > -Iwfiiyx^) 

h=l \i=2 J 

+ Pr (£,»!><)}■ (A.2) 



At this point, we upper bound the RHS of ()A.2|) by the Chernoff bounds, namely, for three 
random variables V, W and Z 

Pr (V > 0) < E [e pV ] , p > (A.3) 
Pr (W < 0, V > 0) < E [ e i w+uV ] , q < 0, u > (A.4) 

Pr(W<0,V>0,Z>0)<E\e tw+sV+kZ ], t<0,s>0,k>0. (A.5) 



18 



The Chernoff versions of the first and last terms in the RHS of l)A.2|) are introduced in ^ 
Eqs.(134)-(137)], and are given by 



vi=2 
n 



^ f £ W > < < \Jh^ e ~ nElic ' p,v) > p * (A - 6) 



v j=2 



where 



and 



£ 2 (c, g, 5, rj) = c | ¥_= + - ln(l - 2g). (A.9) 

l + 2w + (l-2g) A / I ^y 2 

Next, by invoking the Chernoff bound (|A.5|) . we get an exponential upper bound on the second 
term in the RHS of (|48|). Using the notation 



W n(n — w) 



we get (see Appendix IA . 21 for details) 

/ n N 



,i=2 



< ll 2 _L e -g{c,tXs^h,n) ^ _1_ < t <0 7 k>0,s>0 (A.ll) 

1 + 2trj 2rj 



where 



2 



g(c,t,k,s,r),h,n) 



A 



2(1 + 2*7?) 

\ 2 



I - ln(l - 2t) - nr(-) (A.12) 



2(1 - 2£) 2(1-2*) 2 v ' v n y 



and 



A A h 



n — h 

The next step is to find optimal values for k and s in order to maximize the function g. If A;* = 
then the exponent of ift(C) is identical to that of the TSB. In order to find the optimal k > and 
s > which maximize g, we consider the aforementioned probabilities by discussing separately the 
three cases where h < w. h > w and h = w. 
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Case 1: h = w. In this case ( w h = (w w = 1, and we get 



A w Pr (fy^ < r 2 Ui ,u 2 > p w {v x ),v z > -l w , w ^l,^)J < ^i^^AM^) (A.13) 



< t < 0, k > 0, s > 



where 



itijnc + 2V2^c ( s - k ) A w -Al 
g{c,t,k,s,r],w,n) = 



2(1 + 2t7?) 



. 2 



V '"' 7:J A " ^ln(l-2t)-ln(^). (A.14) 



Let us define the parameters 



From IJA.15J) and (|A.16|I . we get 

where 



2(1 -2t) 2(1 — 2t) 2 

£ = a - -/=== (A. 15) 

1 - /9 2 

(£-r)a (A.17) 



« = a/i — : — • ( A - 18 ) 



1 - Ai 



Hence, the Chernoff bounding technique gives 



Pr (J>?< r 2 ^ > ^(^^3 > -VU^2) ] < VrZ^ 6 " 91 ^™'^ (A - 19) 

K i=2 



1 + 2t?7 

1 



< t < 
277 ~~ ~~ 



where 

4t?7nc + 2V2nc^A w - A 2 £ 2 



9i{c,t,£,r,r),h,n) 



2(1 + 2*77) 
r 2 (£ - r) 2 a 2 n 



2(1 — 2t) 2(1-2*) 2 
Maximizing the RHS of (jA~19|) w.r.t. r yields 



+ -ln(l-2t). (A.20) 



dr l-2t l-2t 

>r* = — (A.21) 
1 + cr 
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Notice that < 0, hence plugging r* in (|A.20|) maximizes g\. Substituting r* into ()A,20|) gives 
92(c,t,£,r),w,n) = gi(c,t, £,r* ,7],w,n) 



Atr/nc + 2y/2^c/\ w £ - A 2 W £ 2 if^ 



n 



+ -ln(l-2t). (A.22) 



2(1 + 2^) 2(1 -2t) 2 

A differentiation of g 2 w.r.t. £ and an introduction of the new parameter e = gives 

dg2 V2T~cA w - AjC eg _ 

d£ 1 + 2tr) 1 - 2t 

* _ V^A W (1 - 2t) 

Al(l-2t) + e(l + 2t V )- 



(A.23) 



Again, < 0, so ^* maximizes g 2 - From (|A.21|) . £,* — t* > 0. Since a is non-negative, we get 
that k* in ()A.17|) is not-positive. But since from (jA.llf) . k > 0, this yields that the optimal value 
of k is equal to zero. From the Chernoff bound in (|A.5|) . an optimality of k when it is set to zero 
implies that asymptotically, as n — ► oo 

Pr {^ut < r 2 Ui ,u 2 > PM,u 3 > -l w>w (y u v2) \ = Pr (^v? < r 2 vv v 2 > PM^j ■ (A.24) 



Case 2: h > w. In this case, from (|%0|) it is obvious that p w ^ = J ■ Hence, for this case, 
we get that p w h = ( Wth . From jUl 



2 



4tr/nc + 2^/2^0 s - , fcCro ' h ) A h - A? ( s H " ' 



g(c,t,k,s,rj,h,n) 



2(1 + 2tr?) 

•J l ~(i,h I k 2 n , . , . 

+ -ln(l - 2t) - nr(-). (A.25) 



2(1 -2i) 2(1 — 2t) 2 v ' v n 
In the following, we introduce the parameters 

£ A s _ fo^4 ( A 2()j 

A 



£w,h 

t = k. (A.27) 



Optimization over r yields t* = 0, so k* = 0, and asymptotically (as we let n tend to infinity), 
one gets the following equality in terms of the exponential behaviors: 

Pr (j2 u i ^ r ^ 2 ^ fate)' v * ^ -Uh("i, = Pr (j2 v i ^ r 1v v * ^ Phfa^j • (A.28) 
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Case 3: h < w. From (|4()|). the values of h approve that p Wy h 
that p w h < C w ,h- Define 



A 

T = S 



1 " P W ,h 



kPw,h 



1 " 



From (1X291) and (lA~30l) 

where 



k = -{i-r)a' 



a 



1 - <ft 



t ( (n-fe) > SO We § et fr ° m (|AtlQ|l 

(A.29) 
(A.30) 

(A.31) 
(A.32) 



Cw,h Pw,h 

Since in this case p w h < Cio,fc> then a' > 0. Similarly to the arguments in case 1, we get again that 
the optimal value for k is k* = 0, which implies (|A.28|) in the limit where the block length tends to 
infinity. 

A. 2 Derivation of the Chernoff Bound in (jA.lljl with the Function g in (|A.12|) 

Using the Chernoff bound ()A.5|) and defining 



A, 



w 



n — w 



(A.33) 



we get 



,i=2 



< e ^t(E?=2^ 2 -0+ s (^-/3h(fi))+fc(^+^,h(^i,^)) 

*(E?= 2 ^ 2 -'7(v / 2^-^i) 2 )+s(^2-Ah(v / 2^-^i))+fc I v 3 + 



00 



E 



* < 0, s > 0, fe > 

&w(y/2nc-vi)-p w .hi*. 



tJ2i=2 v'l-tr]v\-2tnr\c+2'r]t\/'lncv\+svi- 



/2nc+sAf l ui+ku3+- 



'( A i»n+P«i ,h v 2) 



E 



"i,h 



(c) 



E [e*^^ 1 ^ E 



tr;i/J+ 2rjt\/2nc+sA h - 



E 



f2 



•E 



-2tnric-sA h V2nc+- 



(A.34) 
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where inequality (a) follows from the Chernoff bound l|A.5|) . equality (b) follows from (|A.1|) . and 
equality (c) follows from the statistical independence of the components of the normalized noise 
vector v. For a zero-mean and unit-variance Gaussian random variable X, the following equality 
holds: 



E 



aX' 2 +bX 



g 2(1— 2a) | 

VI -2a ~ 2' 



Evaluating each term in ()A.34D with the equality in (|A.35|) . and substituting 



(A.35) 



(A.36) 



which follows from ()A.10|) and l)A.33|) . then gives 

n— 3 



E 



E 



1 



t < 



trjvl+ \ 2r]t\/2nc+sA h - 



1-pf 



1 



V 



ll-p 



w,h / J 



E 



E 



tv% + | S 1 ; U2 



\ 

k Pw,h 



2(1+2*77) 



(A.37) 



(A.38) 



vr^2t 



^i- 2 ') , > 0, s > 



vr^ 

From (|A.38|) . straightforward algebra gives 



e 2(l-2t) ) t <0, k>0. 



(A.39) 
(A.40) 



E 



tr)v(+\ 2r]W2nc+sA h - 



ll-P 



1 

w ,h 



-2tnrjc-sA h V2nc+- 



<l- p: 



2 

zu . Ii 



VI + 2ir? 



exp < 
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Plugging (|A.37j) and (|A.39|) - (|A.41|) into (|A.34j) finally gives 
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(A.42) 



1 + 2tr] 2t] 
which proves the Chernoff bound in (|A. with the function g introduced in (|A.12|) . 

A. 3 Monotonicity w.r.t. the Correlation CoefRcient 

Consider the probabilities Pr(i£o->i, -Eq-W' y e C n (6)\zi), and denote the Hamming weights of Cj 
and Cj by di and dj, respectively. In ^7j, it is shown that as long as di > dj, the probabilities 
Pr(£^o-»i! E^py G C n (6)\zi) are monotonically decreasing functions of the correlation coefficients 
p between the planes (o,So,Sj) and (o, So,s,-). Hence, the complex optimization problem in (|24[) is 
simplified by choosing the first error event as well as the complementary error events in the RHS of 
(f2*4"l) to correspond to a codeword with Hamming weight d m i n , and l|2*5|) is obtained. Here we prove, 
that the aforementioned probabilities are monotonically decreasing functions of the correlation 
coefficients for any choice of i,j. As a consequence, one can obtain a version of the ITSB by setting 
in (|24*|) 7Ti = Aj = w where w € {d m i n , . . . , d max }, and choosing the optimal w which minimizes the 
resulting upper bound. In order to prove this, we follow the steps in |17| Appendix I] where it is 
shown that the above probabilities are monotonically decreasing functions of p if 



Z'2 



> p- 



(A.43) 



Note that the joint event (-Eo— >i, y £ C n (8)) implies that the noise component z 2 is in the range 
between f3i(z\) and r Zl (see Fig. so the minimum value of the RHS of (jA.43|l is 



' di{n — dj) 



Clearly, 



Pj(zi) y dj(n - di) 
1 di(n — dj) ^ min(c?j, dj)[n — max((ij, dj) 



> 



, . (A.44) 

y dj (n - di) ^Jdidj (n - di)(n - dj ) 

but from ()34j) . it is evident that the RHS of HA.44|) is the maximal value of p, thus, condition (|A.43|) 
is always satisfied referring to the joint event (Eq^, y S C n (8)). 



References 

[1] D. Divsalar, "A simple tight bound on error probability of block codes with application to Turbo codes," TMO 
progress Report 42-139 NASA, JPL, Pasadena, CA, USA, 1999. 



24 



[2] T. M. Duman and M. Salehi, "New performance bounds for turbo codes," IEEE Trans, on Communications, 
vol. 46, pp. 717-723, June 1998. 

[3] T. M. Duman, Turbo Codes and Turbo Coded Modulation Systems: Analysis and Performance Bounds, Ph.D. 
dissertation, Elect. Comput. Eng. Dep., Northeastern University, Boston, MA, USA, May 1998. 

[4] M. Fossorier, "Critical point for maximum-likelihood decoding of linear block codes," IEEE Communications 
Letters, vol. 9, no. 9, pp. 817-819, September 2005. 

[5] J. Galambos and I. Simonelli, Bonferroni-type inequalities with applications, Springer Series in Statistics, Prob- 
ability and its Applications, Springer- Verlag, New- York, 1996. 

[6] R. G. Gallager, "A simple derivation of the coding theorem and some applications," IEEE Trans, on Information 
Theory, vol. 11, pp. 3-18, January 1965. 

[7] H. Herzberg and G. Poltyrev, "Techniques of bounding the probability of decoding error for block modulation 
structures," IEEE trans, on Information Theory, vol. 40, no. 3, pp. 903-911, May 1994. 

[8] D. Hunter, "An upper bound for the probability of a union," Journal of Applied Probability, vol. 13, pp. 597-603, 
1976. 

[9] G. Poltyrev, "Bounds on the decoding error probability of binary linear codes via their spectra," IEEE Trans, 
on Information Theory, vol. 40, no. 4, pp. 1284-1292, July 1994. 

[10] I. Sason and S. Shamai, "Bounds on the error probability of ML decoding for block and turbo-block codes," 
Annals of Telecommunication, vol. 54, no. 3-4, pp. 183-200, March-April 1999. 

[11] I. Sason and S. Shamai, "Improved upper bounds on the ML decoding error probability of parallel and serial 
concatenated turbo codes via their ensemble distance spectrum," IEEE Trans, on Information Theory, vol. 46, 
no. 1, pp. 24-47, January 2000. 

[12] I. Sason and S. Shamai, "Performance analysis of linear codes under maximum-likelihood decoding: a tuto- 
rial," Foundations and Trends in Communications and Information Theory, vol. 3, no. 1-2, pp. 1-222, NOW 
Publishers, Delft, the Netherlands, July 2006. 

[13] N. Shulman and M. Feder, "Random coding techniques for nonrandom codes," IEEE Trans, on Information 
Theory, vol. 45, no. 6, pp. 2101-2104, September 1999. 

[14] M. Twitto, I. Sason and S. Shamai, "Tightened upper bounds on the ML decoding error probability of binary 
linear block codes," submitted to the IEEE Trans, on Information Theory, February 2006. [Online]. Available: 
|http : //arxiv . o rg/abs/cs . IT/0607003 

[15] S. Yousefi and A. Khandani, "Generelized tangential-sphere bound on the ML decoding error probability of linear 
binary block codes in AWGN interference," IEEE Trans, on Information Theory, vol. 50, no. 11, pp. 2810-2815, 
November 2004. 

[16] S. Yousefi and A. K. Khandani, "A new upper bound on the ML decoding error probability of linear binary block 
codes in AWGN interference," IEEE Trans, on Information Theory, vol. 50, no. 12, pp. 3026-3036, December 
2004. 

[17] S. Yousefi and A. Mehrabian, "Improved tangential-sphere bound on the ML decoding error probability of linear 
binary block codes in AWGN interference," Proceedings 37 th Annual Conference on Information Science and 
Systems (CISS 2005), John Hopkins University, Baltimor, MD, USA, March 16-18, 2005. 

[18] S. Yousefi, "Gallager first bounding technique for the performance evaluation of maximum-likelihood decoded 
linear binary block codes," IEE Proceedings on Communications, vol. 153, no. 3, pp. 317-332, June 2006. 

[19] J. Zangl and R. Herzog, "Improved tangential-sphere bound on the bit error probability of concatenated codes," 
IEEE Journal on Selected Areas in Communications, vol. 19, no. 5, pp. 825-837, May 2001. 



25 




Figure 2: (a): so is the transmitted vector, z\ is the radial noise component, Z2 and z ; 3 are two (not 
necessarily orthogonal) noise components, which are perpendicular to z\, and lie on planes Pi and 
P2, respectively. The doted and dashed areas are the regions where E± and E? occur, respectively, 
(b): A cross-section of the geometry in (a). 
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Figure 3: Comparison between the error exponents for random block codes which are based on 
the union bound (UB), the tangential-sphere bound (TSB) of Poltyrev [5] (which according to 
Theorem n is identical to the error exponents of the ITSB and the AHP bounds), and the random 
coding bound (RCE) of Gallager 6 . The upper and lower plots refer to code rates of 0.5 and 
0.9 bits per channel use, respectively. The error exponents are plotted versus the reciprocal of the 
energy per bit to the one-sided spectral noise density. 
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